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REMARKS 



The present amendment makes editorial changes and corrects typographical errors in the 
specification, which includes the Abstract, in order to conform the specification to the 
requirements of United States Patent Practice. No new matter is added thereby. 

Attached hereto is a marked-up version of the changes made to the specification and 
claims by the current amendment. The attached page is captioned " Versions with Markings to 
Show Changes Made." 

In addition, the present amendment cancels original claims 1-13 in favor of new claims 
14-26. Claims 14-26 have been presented solely because the revisions by crossing out 
underlining which would have been necessary in claims 1-13 in order to present those claims in 
accordance with preferred United States Patent Practice would have been too extensive, and thus 
would have been too burdensome. The present amendment is intended for clarification purposes 
only and not for substantial reasons related to patentability pursuant to 35 U.S.C. §§101, 102, 
103 or 112. Indeed, the cancellation of claims 1-13 does not constitute an intent on the part of 
the Applicants to surrender any of the subject matter of claims 1-13. 
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VERSION WITH MARKINGS TO SHOW CHANGES MADE 



In the Specification: 

The Specification of the present application, including the Abstract, has been amended as 



Thr ee Dim e nsional Mod e l Proc e ssing Apparatus, Thr ee Dim e nsional Mod e l Proc e ssing M e thod 
and Program Providing Medium TITLE OF THE INVENTION 
T e chnical Field THREE-DIMENSIONAL MODEL PROCESSING APPARATUS, THREE- 
DIMENSIONAL MODEL PROCESSING METHOD AND PROGRAM PROVIDING 

MEDIUM 

This invention r e lat e s to a BACKGROUND OF THE INVENTION 
This present invention generally relates to three-dimensional model processing 
apparatus. . More specifically, the present invention relates to a three-dimensional model 
processing apparatus, a three-dimensional model processing method and a program providing 
medium for impl e m e nting proc e ssing such as d e formation, coloring e tc. to thr ee dim e nsional 
model display e d on display of p e rsonal comput e r, e tc.. and mor e particularly to a implementing 
three-dimensional model processing,__ apparatus, a thr ee dim e nsional mod e l proc e ssing m e thod 
and a program providing m e dium which can chang e various attribut e s such as shap e or color, 
e tc. of thr ee dim e nsional mod e l display e d on display in accordanc e with op e ration of op e rator to 
display it as th e r e sult of th e fact that op e rator op e rat e s obj e ct tool as virtual obj e ct 
corr e sponding to thr ee dim e nsional mod e l display e d on display and various e diting tools such as 
a shap e chang e tool, a coloring tool, e tc. 

Background Art 

In the field of computer graphics, thece various techniques have been developed and 
realized various t e chniqu e s with respect to apparatuses or methods for displaying a three- 
dimensional graphic on a display to execute chang e processing ef to modify the orientation ef a 
shape, eter or the like of the displayed three-dimensional graphic. As display t e chniqu e onto 
display of thre e dim e nsional obj e ct, th e r e is One such technique includes a method of inputting 
shape data of an object including depth data by using, ergr for example , a mouse or a two- 
dimensional tablet^-ete. Further, there is also used a method of acquiring data of actual three- 



follows: 



DESCRIPTION SPECIFICATION 



27 



Appl. No. PCT/JP01/03328 




dimensional objects by a three-dimensional digitizer or a three-dimensional scanner to acquire 
display data on th e basic of that data to display such dioplay data on display . In addition, as a 
method of carrying out processing such as deformationreter of an object displayed on a display, 
there is known a configuration using an input unit compris e d of composed of a glove type 
manipulator of th e glov e typ e. 

An example of the configuration of a shape change processing apparatus of display obj e ct 
using the manipulator of using the glove type manipulator is shown in FfGr Figure 1 . In FtGr 
Figure 1, an operator fits or attach e s dons a head mount mounted display 10 to observe a» a 
display object 30 on display . Further, a glove type manipulator 1 1 of th e glov e typ e is fitted over 
or attached at hand of op e rator, to the operator's hand. 

Pressure s e nsor sensors or magnetic s e nsor sensors for detecting mov e m e nt of 
movements of the operator's hand and fing e r is fingers are attached at to the manipulator 1 1 , and 
mov e m e nt of hand of op e rator is d e t e ct e d . A detection signal is inputt e d input to a control unit 
20 through an I/O interface 22. A CPU 21 executes processing in accordance with a program 
stored in a ROM 24 or a RAM 25, and executes chang e processing , e tc. of to modify display 
param e t e r parameters relating to the display object 30 in accordance with the detection signal te 
g e n e rat e n e w thr ee dim e nsional shap e data to display, on th e h e ad mount display 10, thr ee 
dim e nsional obj e ct 30 bas e d on n e w display param e t e r . Display object 30 is updated based on 
the new display parameters through display control means 23. 

For example, in th e cas e wh e re when an operator executes an operation to shav e' 'shave" 
the side surface of the thr ee dim e nsional display object 30 on display by using the glove type 
manipulator 1 1 of th e glov e typ e ?3 _a sensor ef attached to the manipulator 1 1 detects the shaving 
operation of the manipulator 11, and the CPU 21 changes the display param e t e r of th e display e d 
parameters relating to the display object 30 on the basis of d e t e ction signal inputt e d through the 
I/O 22 to display, on th e h e ad mount display 10, display e d obj e ct in which shav e d portion 31 is 
form e d, the detection signal input through the I/O interface 22. Once the display object 30 is 
updated, the head mounted display 10 displays the display object 30 including the shaved portion 
3L 

As stated above, in conventional three-dimensional information input modeling systems, 
various input techniques are used. In the a previously described processing system using mous e s 
a mouse or a two-dimensional tabl e ts, sinc e such tablet, processing is input proc e ssing by using 
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two-dimensional information , th e r e e xist r e striction and/o r . Therefore, a feeling of restriction 
and/or disagreement may exist with respect to the operation of the three-dimensional object. 
Further, various processing such as movement, deformation, cutting, eter and the like with 
respect to the displayed object must be carried out by using only a single editing tool, erg^ 
mous e , r e sulting in th e probl e m that op e rator is difficult to intuitiv e ly grasp for example, a 
mouse. This may result in the operator having difficulty intuitively grasping the proper setting of 
the editing tool. 

Inputs by Input data from a three-dimensional digitizer and/or a three-dimensional 
scanner are useful for input of entering the shape of virtual objects, but are not suitable for 
deformation processin g, e tc. of inputted objects. Such input units ar e e xp e nsiv e . 

Input proc e ssing by the manipulator of th e glov e type is input proc e ssing which can b e 
intuitiv e ly proc e ss e d by and the like of three-dimensional objects. Furthermore, such input units 
are very expensive. 

It is desirable that input processing resulting from manipulation of a glove type 
manipulator is intuitive to the operator. However, in practice, any initial s e tting initialization 
processing is required before actual manipulation processing , e .g., carrying out may be carried 
out. For example, a "push" operation or a "pull" operation, with respect to the object, 
carrying out "pull" op e ration with r e sp e ct ther e to, and th e r e is th e r e for e th e drawback that 
handling is may be required to initialize the settings of the glove type manipulator. This may 
result in a drawback that operating the glove type manipulator may be difficult for a user who is 
unfamiliar with the glove type units. Th e s e Moreover, these glove type input units are also 
consid e rably e xp e nsiv e , very expensive. 

Disclosure of th e Inv e ntion SUMMARY OF THE INVENTION 

This inv e ntion has b e en mad e in vi e w of th e probl e ms of th e abov e d e scribed prior art, 
and its obj e ct is to provid e An advantage of the present invention is that it provides a three- 
dimensional model processing apparatus, a three-dimensional model processing method and a 
program providing medium in which for storing instructions for implementing a three- 
dimensional model processing method in which an operator operates respective tools in a form 
closer to more practical processing by using virtual objects and various tools corresponding to 
respective processing, thereby making it possible to execute various processing of objects such 
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as shape change, surface coloring, e tc. with r e sp e ct to tho thr ee dim e nsional by display e d 
obj e cts. 

This inv e ntion has b ee n mad e in vi e w of th e above m e ntion e d task, and includ e s a 
display unit for thr ee dim e nsionally display e d obj e cts to be e dit e d, an obj e ct tool in which 
mov e m e nt and ori e ntation chang e can b e mad e , an e diting tool in which r e lativ e distanc e with 
rospoct to th e obj e ct tool can b e chang e d, and proc e ssing m e ans for d e t e cting relativo position 
b e tw ee n th e obj e ct tool and th e e diting tool to e x e cut e proc e ssing s e t in corr e spond e nce with th e 
editing tool on th e basis of th e d e t e ct e d r e lativ e positional information to chang e attribut e 
information of the obj e ct to b e e dit e d display e d on th e display unit. 

In this inv e ntion, th e proc e ssing m e ans furth e r includ e s configuration for det e cting 
position information of th e chang e d obj e ct tool bas e d on mov e m e nt and orientation chang e of th e 
obj e ct tool to chang e attribut e information of th e obj e ct to b e e dit e d display e d on the display unit 
on th e basis of th e d e t e ct e d position information. 

Furth e r, th e proc e ssing m e ans includ e s configuration which can execut e plural diff e rent 
proc e ssing corr e sponding to th e kind of e diting tools. 

Furth e r, th e proc e ssing m e ans includ e s configuration which e x e cut e s proc e ssing s e t in 
correspond e nc e with editing tool on th e basis of r e lativ e position information including at l e ast 
on e of r e lativ e distanc e b e tw ee n th e obj e ct tool and th e e diting tool and relativ e angl e b e tw e en 
th e obj e ct tool and th e e diting tool. 

Furth e r, attribut e information of obj e cts to b e e dit e d that th e proc e ssing m e ans chang e s is 
e ith e r on e of attribut e information r e lating to shap e , color and sound of th e obj e ct to b e e dit e d 
display e d on th e display unit. 

Furth e r, th e proc e ssing m e ans includ e s configuration for e x e cuting functional op e ration 
of th e obj e ct to b e edited display e d on th e display unit as proc e ssing s e t in corr e spond e nc e with 
th e e diting tool. 

Furth e r, this inv e ntion is direct e d to a thr e e dim e nsional mod e l proc e ssing m e thod for 
e x e cuting various proc e ssing with r e sp e ct to th e obj e ct to b e e dit e d thr ee dim e nsionally 
display e d on a display unit, and this m e thod includ e s a st e p of d e t e cting r e lativ e position 
b e tw ee n obj e ct tool in which mov e m e nt and ori e ntation chang e can b e mad e and e diting tool in 
which r e lativ e distanc e with r e sp e ct to the obj e ct tool can b e chang e d, and a st e p of e x e cuting 
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proceooing s e t in corr e spond e nc e with th e e diting tool on th e baoio of th e d e t e ct e d rolativ e 
position information to chang e attribute information of th e obj e ct to b e e dit e d on display. 

This inv e ntion furth e r includ e s a st e p of d e tocting position information of the chang e d 
obj e ct tool bas e d on mov e ment and ori e ntation chang e of th e obj e ct tool, and a stop of changing 
attribut e information of th e obj e ct to b e e dit e d display e d on th e display unit on th e basis of th e 
det e ct e d position informati ea? 

This inv e ntion furth e r includ e s a st e p of discriminating a kind of th e e diting tool, and 
serv e s to e x e cut e proc e ssing corr e sponding to th e kind of tho discriminat e d e diting tool. 

Furth e r, at th e st e p of d e t e cting r e lativ e position, r e lativ e position including at l e ast e ith e r 
on e of r e lativ e distanc e b e tween th e object tool and th e e diting tool and r e lativ e angl e b e tw ee n 
obj e ct tool and e diting tool 

Furth e r, attribut e information is e ith e r on e of attribut e information relating to shap e , color 
and sound of obj e ct to b e e dit e d display e d on display m e ans. 

In this inv e ntion, functional op e ration of tho obj e ct to b e e dit e d display e d on the display 
unit is e x e cuted as proc e ssing s e t in correspond e nc e with th e e diting tool. 

In addition, this invention is dir e ct e d to a program providing m e dium for providing 
computer program for e x e cuting, on comput e r syst e m, thr ee dim e nsional model proc e ssing 
which e x e cut e s various proc e ssing with r e sp e ct to th e obj e ct to b e e dit e d thr ee dim e nsionally 
display e d on a display unit, wh e r e in th e comput e r program r e cord e d with r e sp e ct to this 
r e cording m e dium includ e s a st e p of d e t e cting relativ e position b e tw ee n object tool in which 
mov e m e nt and orientation chang e can b e mad e and th e e diting tool in which rolative distanc e 
with r e spect to tho object tool can b e chang e d, and a st e p of e x e cuting processing set in 
corr e spond e nc e with th e e diting tool on tho basis of th e d e t e ct e d r e lativ e position information to 
chang e attribut e information of th e object to b e e dit e d displayed on display m e ans. 

Th e program providing medium adapted so that comput e r program is r e cord e d is m e dium 
for providing, in a form such that computer can r e ad, comput e r program with r e sp e ct to, e .g., 
wid e ly us e d comput e r syst e m which can execut e various program cod e s. As th e m e dium, ther e 
ar e r e cording m e dium such as CD, FD or MO, e tc., and transmission m e dium such as n e twork, 
e tc. and their forms ar e not particularly limit e d. 

Such program providing m e dium d e fin e s coop e rativ e r e lationship in structur e or function 
betw ee n th e comput e r program and th e providing m e dium for realizing function of a 
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pr e d e t e rmin e d computer program on comput e r system. In oth e r words, by inotalling comput e r 
program into th e comput e r system through th e providing m e dium, th e coop e rativ e action is 
e xhibit e d in th e computer system. 

Still furth e r objects of this inv e ntion and more practical m e rits obtainod by this inv e ntion 
will b e com e mor e appar e nt from th e d e scription of embodim e nts which will bo given b e low. 

Bri e f D e scription of th e Drawings 

PIG: — 1 is a block diagram for e xplaining conv e ntional three dim e nsional mod e l 
proc e ssing configuration using glov e typ e manipulator. 

FIG. 2 is a block diagram showing th e configuration of a thr ee dim e nsional mod e l 
proc e ssing apparatus according to this inv e ntion. 

FIG. 3 is a vi e w for e xplaining outlin e of proc e ssing in th e thr ee dim e nsional mod e l 
proc e ssing apparatus according to this invention. 

FIG. 1 is a vi e w showing an e xampl e of th e configuration in which a tabl e t is us e d in 
input unit in th e thr ee dim e nsional mod e l proc e ssing apparatus according to this inv e ntion. 

FIG. 5 is a viow showing an e xampl e of th e configuration in which magn e tic s e nsors ar e 
us e d in input unit in th e thr ee dim e nsional mod e l proc e ssing apparatus according to this 
inv e ntion. 

FIG. 6 is a vi e w showing an e xample of th e configuration in which ultrasonic sensors ar e 
used in input unit in th e throo dimensional mod e l proc e ssing apparatus according to this 
inv e ntion. 

FIG. 7 is a vi e w showing an e xampl e of th e configuration in which ultrasonic 
transpond e rs are used in input unit in th e thr ee dim e nsional mod e l proc e ssing apparatus 
according to this inv e ntion. 

FIG. 8 is a view showing, as flowchart, processing in th e thr ee dim e nsional mod e l 
proc e ssing apparatus according to this inv e ntion. 

FIG. 9 is a viow showing, as flowchart, procossing in th e cas e wh e re a trowel is us e d as 
th e e diting tool in tho thr e e dim e nsional mod e l processing apparatus according to this invention. 

FIG. 10 is a viow showing, as data flow diagram, proc e ssing in th e cas e wh e r e a trow e l is 
us e d as th e e diting tool in th e thr ee dim e nsional mod e l proc e ssing apparatus according to this 
inv e ntion. 
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FIG. 1 1 is a vi e w Ghowing, ao flowchart, processing in th e cas e wh e r e a pinch is us e d as 
th e e diting tool in th e thr ee dim e nsional mod e l proc e ssing apparatus according to this inv e ntion. 

FIG. 12 is a vi e w showing, as data flow diagram, proc e ssing in th e cas e wh e re a pinch is 
us e d as th e e diting tool in th e thr e e dim e nsional mod e l proc e ssing apparatus according to this 
inv e ntion. 

FIG. 13 is a view showing, as flowchart, processing in th e cas e wh e r e a brush is us e d as 
the e diting tool in th e thr ee dimensional mod e l proc e ssing apparatus according to this invention. 

FIG. 11 is a view showing, as data flow diagram, proc e ssing in th e cas e wh e r e a brush is 
us e d as the e diting tool in th e thr e e dim e nsional mod e l proc e ssing apparatus according to this 
inv e ntion. 

FIG. 15 is a vi e w showing, as flowchart, processing in th e cas e wh e r e a spray is us e d as 
th e editing tool in th e thr ee dim e nsional mod e l proc e ssing apparatus according to this inv e ntion. 

FIG. 16 is a vi e w showing, as data flow diagram, proc e ssing in th e cas e wh e r e a spray is 
us e d as th e e diting tool in th e thr ee dim e nsional mod e l proc e ssing apparatus according to this 
inv e ntion. 

FIG. 17A shows action ar e a in th e cas e wh e r e a spray is used as th e e diting tool in th e 
thr ee dim e nsional mod e l proo e ssing apparatus according to this inv e ntion, and FIG. 17B is a 
vi e w showing coloring rang e in th e case wh e re spray is us e d as e diting tool. 

FIGS. 18 A, 18B and 18C ar e views for e xplaining mor e practical e xampl e s of th e 
configuration using th e thr ee dim e nsional mod e l proc e ssing apparatus according to this 
inv e ntion. 

FIG. 19 is a vi e w for e xplaining a mor e practical e xampl e of the configuration using th e 
throe dimensional mod e l proc e ssing apparatus according to this invention. 

B e st Mod e for Carrying Out th e Inv e ntion 

This inv e ntion will now b e d e scrib e d with r e f e r e nc e to th e attach e d drawings. 

FIG. 2 shows a block diagram showing an e mbodim e nt of a thr ee dim e nsional mod e l 
proc e ssing apparatus according to this inv e ntion. Th e thr ee dim e nsional mod e l processing 
apparatus 100 of this e mbodim e nt includ e s c e ntral proc e ssing unit (CPU) 101 for e x e cuting 
proc e ssing program, program m e mory 102 into which proc e ssing program is stor e d, data 
m e mory 103 in which proc e ssing data and information such as attribute and the like. 
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To this end, in an embodiment according to the present invention, a three-dimensional 
model processing apparatus is provided. The three-dimensional model processing apparatus 
includes a display unit for three-dimensionallv displaying an object and an object tool 
representing the displayed object, wherein the object tool is capable of three-dimensional 
movement such that the position and the orientation of the object tool can be changed. Also 
included are an editing tool in which relative distance with respect to the object tool can be 
changed and a processor for detecting the relative position between the object tool and the 
editing tool. Preferably, the processor is configured to execute processing determined by the 
editing tool to modify attribute information of the displayed object on the basis of the detected 
relative positional information. 

In an embodiment, the processor is further configured to detect position information of 
the object tool based on changes in the position and the orientation of the object tool Preferably, 
this detected position information is used to modify the attribute information of the displayed 
object. In an embodiment, the processor is further configured to execute processing 
corresponding a number of different types of editing tools. In an embodiment, the detected 
relative position information includes at least one of the relative distance between the object tool 
and the editing tool and the relative angle between the object tool and the editing tool. 
Preferably, the attribute information of the displayed object is attribute information relating to at 
least one of shape, color and sound. Further, the processor is configured to execute functional 
operation of the displayed object as processing determined by the editing tool. 

In another embodiment according to the present invention, a three-dimensional model 
processing method is provided. Also provided are an object that is three-dimensionally 
displayed on a display unit, an object tool representing the displayed object and an editing tool. 
The object tool is capable of three-dimensional movement such that the position and the 
orientation of the object tool can be changed and the editing tool is capable of three-dimensional 
movement such that the relative position between the editing tool and the object tool can be 
changed. The three-dimensional model processing method includes the steps of detecting the 
relative position between the object tool and the editing tool and executing processing 
determined by the editing tool to modify attribute information of the displayed object on the 
basis of the detected relative position information. 
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In an embodiment, the three-dimensional model processing method further includes the 
steps of detecting position information of the changed object tool based on chanees in the 
position and the orientation of the object tool and modifying the attribute information of the 
displayed object based on the detected position information. In an embodiment, the three- 
dimensional model processing method further includes the steps of selecting the editing tool 
from a plurality of different types of editing tools and executing processing corresponding to the 
type of editing tool selected. 

In an embodiment, detected relative position information is at least one of the relative 
distance between the object tool and the editing tool and the relative angle between object tool 
and editing tool. In an embodiment, the attribute information of the displayed object is attribute 
information relating to at least one of shape, color and sound. In a further embodiment, the 
three-dimensional model processing method includes the step of executing functional operation 
of the displayed object as processing determined by the editing tool. 

In yet another embodiment according to the present invention, a computer readable 
medium storing a computer readable program is provided. Also provided are an object that is 
three-dimensionally displayed on a display unit, an object tool representing the displayed object 
and an editing tool. The object tool is capable of three-dimensional movement such that the 
position and the orientation of the object tool can be changed and the editing tool is capable of 
three-dimensional movement such that the relative position between the editing tool and the 
object tool can be changed. The computer readable program includes the steps of detecting the 
relative position between the object tool and the editing tool and executing processing 
determined by the editing tool to modify attribute information of the displayed object on the 
basis of the detected relative position information. 

Additional features and advantages of the present invention are described in, and will be 
apparent from, the following Detailed Description of the Invention and the Figures. 

BRIEF DESCRIPTION OF THE FIGURES 

Figure 1 is a schematic diagram showing a conventional three-dimensional model 
processing configuration using a glove type manipulator. 

Figure 2 is a schematic diagram showing an embodiment of a three-dimensional model 
processing apparatus according to the present invention. 
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Figure 3 is a schematic diagram illustrating processing in a three-dimensional model 
processing apparatus according to an embodiment of the present invention. 

Figure 4 is a block diagram showing an embodiment in which a tablet is used as an input 
unit in a three-dimensional model processing apparatus according to the present invention. 

Figure 5 is a block diagram showing an embodiment in which magnetic sensors are used 
in an input unit in a three-dimensional model processing apparatus according to the present 
invention. 

Figure 6 is a block diagram showing an embodiment in which ultrasonic sensors are used 
in an input unit in a three-dimensional model processing apparatus according to the present 
invention. 

Figure 7 is a block diagram showing an embodiment in which ultrasonic transponders are 
p used in an input unit in a three-dimensional model processing apparatus according to the present 
; s =4 invention. 

Figure 8 is a flowchart illustrating processing in a three-dimensional model processing 

m 

£|i a pparatus according to an embodiment of the present. 

LH; Figure 9 is a flowchart illustrating processing where a trowel is used as an editing tool in 

Q a three-dimensional model processing apparatus according to an embodiment of the present 
p J; invention. 

C(| Figure 10 is a data flow diagram illustrating processing where a trowel is used as an 

t 5s Ji 

jljj editing tool in a three-dimensional model processing apparatus according to an embodiment of 
the present invention. 

Figure 1 1 is a flowchart illustrating processing where a pinch is used as an editing tool in 
a three-dimensional model processing apparatus according to an embodiment of the present 
invention. 

Figure 12 is a data flow diagram illustrating processing where a pinch is used as an 
editing tool in a three-dimensional model processing apparatus according to an embodiment of 
the present invention. 

Figure 13 is a flowchart illustrating processing where a brush is used as an editing tool in 
a three-dimensional model processing apparatus according to an embodiment of the present 
invention. 



36 



AppL No. PCT/JPO 1/03328 




Figure 14 is a data flow diagram illustrating processing where a brush is used as an 
editing tool in a three-dimensional model processing apparatus according to an embodiment of 
the present invention. 

Figure 15 is a flowchart illustrating processing where a spray is used as an editing tool in 
a three-dimensional model processing apparatus according to an embodiment of the present 
invention. 

Figure 16 is a data flow diagram illustrating processing where a spray is used as an 
editing tool in a three-dimensional model processing apparatus according to an embodiment of 
the present invention. 

Figure 17A is a perspective view showing an action area in an embodiment where a spray 
is used as an editing tool in a three-dimensional model processing apparatus according to the 
present invention, and Figure 17B is a perspective view showing a coloring range in an 
embodiment where spray is used as an editing tool 

Figures 18A. 18B and 18C are perspective views showing embodiments of a three- 
dimensional model processing apparatus according to the present invention. 

Figure 19 is a perspective view showing an embodiment of a three-dimensional model 
processing apparatus according to the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

Figure 2 is a schematic diagram showing an embodiment of a three-dimensional model 
processing apparatus according to the present invention. The three-dimensional model 
processing apparatus 100 includes a central processing unit (CPU) 101 for executing a 
processing program, a program memory 102 into which a processing program is stored, and a 
data memory 103 in which processing data and information such as attribute data of position, 
orientation, shape, color, eter and the like of the object to be edited or attribute data of the editing 
tool, eter and the like are stored ?. Also included are a frame memory 104 in which picture 
information for displaying, on a picture display unit 1 10, the object to be edited, the editing tool 
aftd, an instruction to user, eter and the like are stored, an input unit 105 for inputting various 
instructions with respect to the object to be edited, an external storage unit 106 for storing 
observation information or a processing result, and a bus 107 for permitting data transfer 
between th e s e the respective units. The picture display unit 1 10 is connected through the frame 
memory 104. Thus, picture data stored in the frame memory 104 is may be displayed. 
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The input unit 105 which io charact e riz e d m e ans in this invention is constituted by plural 
groups includes a number of tools for carrying out various processing such as deformation, 
surface color change, eter and the like with respect to the object to be edited having thre e 
dim e nsional shap e display e d on th e pictur e display unit 110. (i.e., the object being displayed on 
the display unit or the displayed object). 

Prior to moro practical d e scription of r e sp e ctiv e m e ans of FIG. 2, th e concept of this 
invontion will b e d e scribed with r e f e r e nc e to FIG. 3. Th e A three-dimensional model processing 
apparatus of this according to an embodiment of the present invention includes, as shown in HGr 
Figure 3, an object tool 301 corresponding to the displayed object to b e e dited and editing tools 

302 to 306 for implementing various processing to the obj e ct to b e e dit e d, which ar e display e d 
on display displayed object. The object tool 301 and the editing tools 302 to 306 are also 
displayed on a display 307 . By using these tools, processing such as deformation, coloring, eter 
with r e sp e ct to object 308 to b e e dit e d display e d on display 307 is e x e cut e d, and the like with 
respect to displayed object 308 may be executed. 

In FIG. 3. as editing tools, th e r e ar e show As shown in Figure 3, the editing tools 302 to 
306 include a push tool 302 having, ergr? for example, having the shape of "trow e l" a trowel for 
executing a "push" operation with respect to the displayed object to b e e dit e d, 308; a pull tool 

303 having the shape of a pinch or tongs for executing a "pull" operation with respect to the 
displayed object to b e e dit e d, 308; a brush tool 304 having, erg^ for example, the shape of 
"brush" a brush for executing a "fine coloring" operation with respect to the displayed object te 
b e e dit e d, 308; a spray tool 305 having, erg^ for example, the shape of "color sprav" a can of 
spray paint for executing a "rough coloring" operation with respect to the displayed object to b e 
e dit e d, 308; and a cut tool 306 having, erg^ for example, the shape of "knif e " a knife for 
executing a "cut off, separate" operation with respect to the obj e ct to b e e dit e d, displayed object 
308. 

As th e r e sult of the fact that op e rator op e rat e s various tools shown in FIG. 3 The operator 
operates the editing tools 302 to 306 in a predetermined action area #e.g., a sensor detection 
area) as indicated by action areas 201 to 204 in FIG. 3, in mor e practical s e ns e , carries out 
chang e of Figure 3, thereby causing a change in distance between the object tool 301 and the 
editing tools 302 to 306 and a change ef in the angle therebetween . As a result , processing 
corresponding to the respective tools, that is , deformation of the displayed object to b e edited 
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308 corresponding to the "push" operation by the push tool 302 and the "pull" operation by the 
pull tool 303, and/or change of the surface color of the displayed object to bo e dited 308 
corresponding tojhe brush tool 304 and the spray tool 305 are implemented to obj e ct 308 to b e 
e dit e d display e d on display 307. the displayed object 308. 

Ag m e taphor of clay mod e l subj e ct to oporation ouch as d e formation or coloring, e tc., 
input information ft em -In an embodiment, a clay model is used as the object tool 301 and is 
subjected to operations such as deformation, coloring, or the like. Information is input from a 
sensor for detecting the position and the orientation of the clay model in the three-dimensional 
space is us e d. This io clay obj e ct s e nsor, and tool 301 for obj e ct of FIG. 3 . The object tool 301 
may be a three-dimensional shape as shown in FiGr Figure 3, for example, a spherical shape or a 
shape approximate to the obj e ct to b e e dit e d displayed on display. This object 308. The object 
tool 301 has th e configuration is configured such that the operator can hold it by one hand se 
such that it ea» may be freely moved or rotated. 

Mor e ov e r, editing tool m e taphor for carrying out d e formation or coloring with r e sp e ct to 
th e obj e ct to b e e dit e d also has, as param e t e r, input information from The editing tool includes, 
as a parameter, information input from a sensor for detecting the position and the orientation of 
the editing tool in the three-dimensional space. The editing tool has a shape that operators can 
intuitively associate with a processing mode in accordance with the respective processing. 
Nam e ly, as shown in FIG. 3, th e r e ar e shap e of "trow e l" as push tool 302. shap e of "pinch" as 
pull tool 303, shap e of "brush" as brush tool 301, shap e of "spray" as spray tool 305 and shap e of 
"knif e " as cut tool 306, e tc. For example, as shown in Figure 3, the editing tool can be in the 
shape of a trowel as a push tool 302, the shape of a pinch or tongs as a pull tool 303, the shape of 
a brush as a brush tool 304, the shape of a can of spray paint as a spray tool 305, and the shape of 
a knife as a cut tool 306. 

Th e s e The object tool 301 and the editing tools 302 to 306 correspond to the input unit 
105 in FIG. 2, and mor e practical e xample of th e configuration of this Figure 2. An embodiment 
of the input unit will be described with reference to FIGS. Figures 4 to 7. It is to be noted that, in 
FIGS. Figures 4 to 7, the editing tools 302 to 306 are describ e d in th e stat e roughly divided into a 
tool for deformation processing and a tool for color processing without fin e ly classifying th e m. 
the type of tool being used. 
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HGr Figure 4 is an e xampl e embodiment where an input unit is constitut e d with includes 
a tablet 404. Position information and orientation information of an object tool 401, a 
deformation tool 402 and a coloring tool 403 are detected by input through the input tablet 404? 
afld, A detection data ar e inputt e d. D e t e ction function by a chip ef ^_a coil , or the like is added to 
the object tool 401, the deformation tool 402 and the coloring tool 403 similarly to an input pen 
associated with a conventional tablek-aft d. Thus, the position and the orientation information of 
the respective tools a*e may be detected. Wh e n For example, coils from which having different 
detection values diff e r e nt from e ach othor are obtain e d are provided on the respective surfaces 1 
to 6 of the object tool 301 of hexahedron as shown in FIG. 3, for e xampl e , mounting ourfac e s on 
the tabl e t ar e discriminat e d, thus shape as shown in Figure 3. Thus, when the different detection 
values of the coils are obtained via the tablet 404, mounting surfaces are identified, thereby 
making it possible to obtain three-dimensional information from these mounting surfaces. 
Alternatively, thete a configuration may be employed a configuration in which Z-axis 
information obtained from other sources, such as an optical sensor e*^ magnetic sensor or the 
like is combined with XY axis (i.e., two-dimensional) information obtained from the tablet te 
th e r e by discriminat e 404 to identify a three-dimensional position and to obtain three-dimensional 
data. 

FIGt Figure 5 is an e xampl e embodiment where an input unit is constitut e d by using 
includes magnetic sensors. The area within a magnetic field produced ftem by a magnetic 
source 504 is caus e d to b e an action area, and operations by an object tool 501, a deformation 
tool 502 and a coloring tool 503 are carried out within the action area. Magnetic sensors are 
respectively attached to the object tool 501, the deformation tool 502 and the coloirng tool 503, 
and th e magn e tic s e nsors of th e obj e ct tool 501, the d e formation tool 502 and th e coloirng tool 
503- coloring tool 503. These magnetic sensors output magnetic displacement data to dedicated 
interface hardware 505 within the action area s e t within magn e tic fi e ld produc e d from the 
magn e tic sourc e 504 . The dedicated interface hardware 505 calculates position information data 7 
and orientation information data of the respective tools on the basis of these this data. It is to b e 
not e d that In an embodiment, the dedicated interface hardware 505 shown in FIGr Figure 5 may 
be replaced by a calculation processing program of software program. 

Figurer 
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FIGr 6 is an e xampl e embodiment where the an input unit is constitut e d by uoing includes 
ultrasonic sensors. The area where ultrasonic wav e waves produced from a by an ultrasonic 
transmitting section 604 can be detected is caus e d to b e an action area, and proc e ssing operations 
by an object tool 601, a deformation tool 602 and a coloring tool 603 are carried out within the 
action area. Ultrasonic sensors are respectively attached to the object tool 601, the deformation 
tool 602 and the coloring tool 603 , and th e ultrasonic s e nsors of th e obj e ct tool 601, the 
deformation tool 602 and th e coloring tool 603 . These ultrasonic sensors detect arrival times 
from the source of an ultrasonic wave or the interference of ultrasonic waves and the like within 
the action area which is th e ar e a whore ultrasonic wav e produc e d from th e ultrasonic transmitting 
section 601 can b e det e ct e d d e t e ct arrival tim e s from sourc e of ultrasonic wav e or int e rf e r e nc e of 
ultrasonic wav e , e tc . Dedicated interface hardware 605 calculates position information data and 
orientation information source data of the respective tools on the basis of th e s e the detected data. 
It is to b e not e d that In an embodiment, the dedicated interface hardware 605 shown in FIGr 
Figure 6 may be replaced by a calculation processing program of softwar e , software program. 

FIG. 7 is a s e cond e xampl e wh e r e th e input unit is constitut e d by using ultrasonic s e nsor. 
A Figure 7 is an embodiment where an input unit includes an ultrasonic sensor. An ultrasonic 
transmitting/receiving section 704 carri e s out transmitting and r e c e iving of transmits and 
receives ultrasonic wav e waves . The area where ultrasonic wav e waves produced from by the 
ultrasonic transmitting/receiving section 704 can be detected is caus e d to b e an action area^-aftd 
proc e ssing . Processing operations by an object tool 701, a deformation tool 702 and a coloring 
tool 703 are carried out within the action area. Transponders having a function to receive 
ultrasonic wav e waves produced &em by the ultrasonic transmitting/receiving section 704 and to 
send back it ultrasonic waves are respectively attached at the object tool 701, the deformation 
tool 702 and the coloring tool 703. The ultrasonic transmitting/receiving section 704 receives 
ultrasonic waves sent back from the respective tools to detect tim e r e quir e d for going and 
r e turning of a transmission time of the ultrasonic wav e or waves, interference of the ultrasonic 
wav e , e tc. waves, or the like to calculate position information data and orientation information 
source data of the respective tools on the basis of th e se this data. 

The input unit 105 shown in HGr Figure 2 is realized as an input device using various 
sensors as described above. It is to b e not e d that In an embodiment, information input/output 
with respect to a three-dimensional model processing apparatus 100 from sensors of the 



41 



Appl. No. PCT/JP01/03328 




respective tools may be configuration through configured as a data line of wire or a wireless data 
transmitting/receiving configuration. In addition, ao the configuration of an embodiment, the 
input unit 105 , th e r e may b e e mploy e d include a combination of the various sensors. 

Ao th e external storag e unit 106 shown in FIG. 2, In an embodiment, a random accessible 
memory medium such as a hard disk drive (HDD) or optical disk, e tc. iG d e sirabl e . In addition to 
th e abov e, an optical disk, or the like is desirable as the external storage unit 106 shown in Figure 
2. Additionally , the external memory unit may b e constitut e d by include a tape streamer or non- 
volatile semiconductor memory represented by a memory stick. In addition, Also, it is alse 
possible to utilize an external storage unit of another network-connected ethep system. In an 
embodiment, the The central processing unit (CPU) 101 executes processing in accordance with 
a processing program recorded in the program memory 102. 

Flow The flow of processing using the three-dimensional model processing apparatus 
shown in FIG. 2 will now b e d e scrib e d in accordanc e with th e proc e ssing flow of FIG. 8. Th e 
proc e ssing flow shown in FIG. 8 is Figure 2 will be described with reference to the flowchart 
shown in Figure 8. The flowchart shown in Figure 8 illustrates processing executed in 
accordance with a processing program recorded in the program memory 102. 

In FIG. 8, th e e xampl e of th e cas e wh e r e push tool 302, pull tool 303, brush tool 301 as 
coloring tool and spray tool 305 w r hich hav e b ee n e xplain e d with r e f e r e nc e to FIG. 3 ar e used as 
e diting tool will b e e xplain e d. In addition to th e abov e Figure 8, an embodiment will be 
explained using at least one of a push tool 302. a pull tool 303, a brush tool 304 (i.e.. a coloring 
tool) and a spray tool 305 as an editing tool. In an embodiment , it is possible to use various 
types of editing tools not listed above for executing various types of processing. 

Initially, at step S801 of FIG. 8 , data of a position and an orientation , e tc. on in the three- 
dimensional space of the object to be edited ar e acquir e d. Such data ar e data stor e d in th e data 
m e mory 103 shown in FIG. 2 and corr e spondin g (i.e., the displayed object) are acquired. This 
data corresponds to attribute data such as position, orientation, shape, color, e tc. of the obj e ct to 
b e e dit e d display e d on th e pictur e display unit 110. Initial and the like of the displayed object and 
is stored in the data memory 103 (shown in Figure 2). An initial position of the object tool 301 
{shown in FIG. 3 Figure 3) is set as a position and an orientation corresponding to these this 
attribute data. Thereafter, i» the cas e wh e r e operator moves the object tool 301 within the action 
area which is (i.e.. the area where position detection by a sensor can be made), the position and 
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the orientation of the object tool are detected by a sensor output of the object tool 301, and the 
attribute data of the displayed object is rewritten in accordance with the movement. Thus, the 
obj e ct to b e e dit e d is display e d a ppearance displayed object is updated on the picture display unit 
1 10 on the basis of the rewritten attribute data. 

Then, at step S802, wh e th e r or not tool, i. e ., a check is performed to determine whether 
the object tool and the editing tool ouch as d e formation, and coloring, e tc. ar e includ e d are_within 
the action area is ch e ck e d . The action area is eaeh the detectable area of the various sensors used 
as the input unit. In th e case whor e tabl e t is As discussed above, various input units can be used 
in accordance with various embodiments of the present invention. For example, a tablet can be 
used as the input unit? and the action area is tabl e t the area^-fe of the cas e wh e r e tablet, a 
magnetic sensor is can be used as the input unit? and the action area is the detectable area of the 
magnetic senso r. In th e cas e wh e r e ^ jn; an ultrasonic sensor is can be used as the input unit T and 
the action area is the detectable area of the ultrasonic sensor. 

When it is judg e d determined at step S803 that the tool is not includ e d within the action 
area, processing jumps to step S814. In th e cas e wher e When it is judg e d determined that tool is 
includ e d within the action area, processing proceeds to step S804 to judg e determine the type or 
kind (ID) of tool being used . The judgm e nt determination of the tool kind h e r e iG judgment type 
is determined with respect to the kiad type of editing teei being performed, such as deformation, 
coloring, ete and the like . With respect to the kiad types of these editing tools, the operator 
hims e lf may designate the tool to input identification data (ID) indicating the tool kiad type , or 
may employ a configuration for outputting an identification signal (ID) from the editing tool 
e xisting in within the action area to carry out discrimination of th e identify the type of tool on the 
basis of the identification signal. 

When the kiad type of editing tool is judg e d determined at the step S804, processing 
proceeds to proc e ssing of steps S805 to S812. Th e proc e ssing from th e st e p Step S805 to the 
step S806 is illustrates processing in th e cas e where the editing tool is a push tool, th e proc e ssing 
from th e step S807 to the step S808 is illustrate processing in th e cas e where the editing tool is a 
pull tool, th e proc e ssing from th e step S809 to the step S810 is illustrate processing in th e cas e 
where the editing tool is a brush tool, and th e proc e ssing from th e step S81 1 to the step S812 is 
illustrate processing in th e cas e where the editing tool is a spray tool. The r e sp e ctiv e processing 
of th e illustrated by steps S806, S808, S810, S812 a*e corresponds to subroutines corr e sponding 
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te of the respective tools, and are called at tho timo point when the editing tool is specified and 
ar e e x e cut e d. In th e cas e wh e r e . Where the editing tool does not correspond to aay a known type 
of editing tool, processing proceeds to step S813, at which whereby a warning is producedr-Stteh 
warning is. e .g.. proc e ssing for , for example, displaying a warning message on the picture 
display unit 1 10 shown in FIG. 2, or outputting a warning soundrete. 

Explanation will be given below with reference to FIG. 9 and figur e o {succ ee ding th e r e to 
Figures 9 to 16 in connection with the respective processing of the steps S806, S808, S8IO7 and 
S812 7 (i.e., subroutine processing corresponding to the respective editing tools). 

At step S814, the respective editing tools are displayed on the picture display unit 110, 
and display of the object to b e e dit e d processed by the editing tool is carried out. The display 
modes (form) of the respective editing tools are modes based on shape data set e v e ry e diting tool 
in advance for each editing tool . For example, if the editing tool is a push tool, the shape of 
"trow e l" a trowel is displayed on the picture display unit 110r4f or if the editing tool is a spray 
tool, the shape of ^spfav ^a can of spray paint is displayed on the picture display unit 110. 
Moreover, the displayed object to b e e dit e d is displayed as an object to which deformation e* A 
coloring or the like is applied by each e diting tool. Nam e ly, obj e cts to b e e dited in which 
attributeo of the editing tools. Namely, as attribute data such as position, orientation, shape, 
color, e tc. of th e obj e ct to b e e dit e d chang e d by r e sp e ctiv e e diting tools ar e r e fl e ct e d ar e 
display e d. Attribut e s of obj e ctG to b e e dit e d r e writt e n by op e ration by op e rator of r e sp e ctive 
editing tools ar e and the like of the displayed object is modified by the editing tools, these 
changes are reflected and displayed on the picture display unit HQ. Attribute data of the 
displayed object rewritten by the operator using the editing tools is stored into the data memory 
103 {shown in FIG. 2 Figure 2) . 

Further, at step S815, compl e tion of proc e ssing is judg e d. Compl e tion of proc e ssing may 
b e input from us e r who is operator, or a check is performed to determine if processing is 
completed. Processing may be completed by input from a user who is the operator, or from a 
rule prescribed within the processing application. For example, in th e cas e of with a game 
program, compl e tion of processing may be completed by a completion command by such as 
game over. In addition, judgm e nt of compl e tion bas e d on r e striction of the determination that 
processing is completed may be based on a hardware or software restriction such as memory full? 
e tc. may b e carri e d out. In tho case wh e re it is judg e d at th e and the like. When it is determined 
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at step S815 that processing is not completed, processing returns to the start of the processing 
flow to repeatedly execute similar processing. 

Th e d e tail of tho subroutin e proc e ssing shown in FIG. 8 will now b e d e scrib e d. FIG. 9 io 
a flow for explaining proc e ssing in th e cas e wh e r e push tool is appli e d to th e obj e ct to b e e dited 
to apply d e formation. St e p S901 is Figure 9 is a flowchart illustrating processing using a push 
tool to perform deformation of the displayed object. Step S901 is a step for obtaining 
position/orientation data on the three-dimensional space of "trow e l" a trowel being used as a push 
tool. As previously described, such data are data obtained on the basis of respective sensor 
outputs in accordance with the kiftd type of input unit utilized, i^r for example , tablets, magnetic 
sensors et a ultrasonic sensors or the like. 

hiT 

Th e n, step S902 is proc e ssing for d e t e rminin g , a space relationship/relative position 
between the displayed object to b e e dit e d and the editing tool 6e» is determined based on the 
position/orientation data of the obj e ct to b e e dit e d which has b ee n acquir e d at th e pr e viously 
m e ntion e d st e p S801 of FIG. 8 and displayed object (acquired in step S801 of Figure 8) and the 
position/orientation data of the trowel as the editing tool "trow e l" d e t e rmin e d at th e st e p 
S9Q Uobtained in step S901). 

Th e n, at In step S903, judgm e nt a determination is made as to whether or not deformation 
proc e ssing of th e obj e ct to b e e dit e d using e diting tool "trow e l" is e x e cut e d is carri e d out. In 
mor e practical s e ns e , this judgm e nt s e rv e s to carry out of the displayed object by the trowel as 
the editing has been carried out. Put more simply, step 903 serves to determine whether or not 
position data of the editing teete tool as detected by various sensors are admitt e d into th e obj e ct 
to b e e dit e d overlaps with position data of the object tool (i.e., has the editing tool penetrated the 
displayed object) . It is to be noted that it is not necessary that editing tools 302 to 306 a*e 
actually in th e obj e ct tool 301 in FIG. 3 in this cas e , but th e re may b e e mploy e d configuration to 
e x e cut e proc e ssing which d e termin e s that in th e cas e wh e r e both tools ar e clos e so that th e 
distance th e r e b e tw ee n actually penetrate the object tool 301. The apparatus may be configured 
to suppose that the editing has penetrated the displayed object if the distance between the object 
tool and the editing tool is less than a predetermined value . Alternatively, the , it is judg e d that 
th e e diting tool is admitt e d into th e obj e ct to be e dited. 
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Alternatively, object tool 301 may be constituted with, e.g., composed of, for example, a 
deformable sponge shaped material to also e mploy configuration such that the editing tool eaa 
may actually eatef penetrate the object tool 301. In thio cas e , th e r e may b e e mploy e d a 
configuration to execut e d e formation of th e obj e ct to b e e dited display e d on th e pictur e display 
unit 1 10 on th e basis of an embodiment, deformation of the displayed object may be executed on 
the basis of the actual position data of the editing tool and the object tool in a mann e r caus e d to 
b e in corr e spond e nce with th e actual position data . 

Deformation processing of the object to be edited as displayed on the picture display unit 
110 at step S904 can be realized by moving positions of vertices of polygons when the display 
data is polygons data. In this cas e , in th e cas e wh e r e Where the distance between vertices is 
gr e atly apart large , processing for generating new vertices is executed. Moreover, in the cas e 
where the displayed object to b e e dit e d is represented as parametric curves, their control points 
are moved. Further, in the case where the displayed object to b e e dit e d is represented by voxels, 
deformation can be realized by (Boolean operation) processing to add data. The deformation 
processing is carried out in accordance with data mode (form). In either case, processing based 
on displacement data of the relative distance between object tool and editing tool is carried out. 

After deformation processing of the displayed object to b e e dit e d is executed at th e in 
step S904, the rewritten attribute data of the displayed object to b e e dit e d of including the 
deformed result is stored into the data memory 103 or external storage unit 106 (shown in FIG. 2 
at Figure 2) in step S905. 

In order to supplement the explanation of the flowchart shown in 5?Gr Figure 9, 
explanation will be furth e r giv e n in conn e ction with proc e ssing in th e cas e wh e r e push tool 
"trow e l" is us e d given with reference to the data flow diagram of FIG. 10 Figure 10 using the 
trowel as a push tool . The data flow diagram is a view showing the flow of data extending from 
a data source within the object to a target within oth e r another object via a process for converting 
data. The data flow diagram is constitut e d by includes processes (indicated as ellipses) for 
converting data, data flows (indicated as arrows) for carrying data, actor objects (indicated as 
rectangles) indicating production and consumption of data, and data store objects (indication put 
between two lines) for passively carrying out storage of data. 
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The data flow of HGr Figure 10 is illustrates processing by push tool "trowol", and 
proo e ooing using the trowel as a push tool. Processing is started by operation of the object tool 
(sensor) 1001 as an actor object and the trowel tool (sensor) 1002 as an actor object . 

From the object tool 1001, orientation information and position information of the object 
are acquired. From the trowel tool 1002, position information of the tool is acquired. 

Further, from the shape information of the object which is attribute data of the object to 
be edited (i.e.. the displayed object) which has been acquired in advance? and the surface 
attribute information of the object, the orientation information and the position information of the 
object and the position information of the tool which are stored in the data memory 103 of FIG. 
a(shown in Figure 2) , calculation processing 1003 ef to calculate the degree of deformation is 
executed. On the basis of this calculation, processing for changing the attribute information? 
(i.e., deformation processing 1004) is executed. In combination with these this processing, 
rendering processing 105 as proc e ssing for generating a display picture with respect to the 
picture display unit 1 10 is executed. Thus, a rendering processing result is stored in the memory 
for the display? (i.e., the frame memory 104 of FIG. 2. Figure D and is displayed on the picture 
display unit 110. 

FIG. 1 1 is a flow for e xplaining proc e ssing in th e cas e wh e r e pull tool "pinch" is appli e d 
to th e obj e ct to b e e dit e d to apply d e formation. St e p SI 101 is F igure 1 1 is a flowchart illustrating 
processing using a pinch or tongs as a pull tool to perform deformation of the displayed object. 
Step SI 101 is a step for obtaining position data and orientation data e» in the three-dimensional 
space of ^pifteh ^the pinch or tongs used as the pull tool. As previously described, such data are 
data obtained on the basis of respective sensor outputs in accordance with kiad type of input unit 
utilized , i. e ., tabl e ts, magn e tic sensors and ultrasonic s e nsors. . 

Th e n, at stop SI 102, wh e th e r or not th e In step SI 102, a determination is made as to 
whether or not a switch of the pinch is pushed down is judg e d. Th e pull tool "pinch" of this 
embodim e nt has th e configuration in which th e switch is provid e d at th e pull tool "pinch"^ _Thg 
pinch used a pull tool in this embodiment is configured with a switch for determining the start 
point and the end point of the pull operation to set th e . The position of the pool tool at the time 
point when the switch is pushed down as is either the start point ("ON") or the end point ("OFF") 
of the pull operation with respect to the displayed object to be e dit e d . When it is judg e d at th e 
determined in step SI 102 that the switch is not pushed down, processing for turning OFF "flag to 
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show it was pushed down last" is executed at step SI 109 thereafter to execute processing to 
return to the end position 7 4re7 ? (i.e., the flow start position}. 

When it is judg e d determined at the step SI 102 that the switch has been pushed down, 
processing proceeds to step SI 103, at which point it is determined whether or not "flag to show it 
was pushed down last" is turned ON iG judg e d. In th e case wh e r e . Where "flag to show it was 
pushed down last" is not turned "ON", it is judged that the push down of the switch ef at this 
time is "ON", ir ^i.e., the current position is the start position of the pull operation). At step 
SI 108, processing for storing position information of the pull tool "pinch" of switch-pushing- 
down-position into the data memory 103 to return to the flow start position is executed. 

When it is judg e d determined at the step SI 103 that "flag to show it was pushed down 
last" is turned ON, it is judged determined that the current switch pushing down of this time is 
"OFF" and this time period is the execution period of the pull operation. In this case, at At step 
SI 104, position information of the switch pushing position? (i.e., position information as the end 
position of the pull operation) is stored into the data memory 103. 

At In the subsequent step SI 105, a movement quantity of the pinch used as the pull tool 
"pinch" from the last switch-pushing-down-point to the switch-pushing-down-point of this time is 
calculated. This movement quantity is the effective processing period of the pull operation. At 
In the subsequent step SI 106, deformation processing of the object is executed on the basis of 
the movement quantity determined at the step SI 105. At step SI 107, new attribute information 
obtained by deformation processing? (i.e., shape data of the object is stored into the data 
memoryr). Display of the object is carried out on the picture display unit 1 10 on the basis of the 
new attribute information. 

For suppl e m e nting th e d e ocription In order to supplement the explanation of the 
flowchart shown in ¥iGr Figure 11, explanation will be furth e r given in conn e ction with th e case 
wh e r e th e pull tool "pinch" io us e d given with reference to the data flow diagram of FIG. 12. In 
FIGr Figure 12 using the pinch as a pull tool. In Figure 12, processing is started by operation of 
the object tool (sensor) 1201 as an actor object and pinch tool (sensor) 1202 as an actor object . 

From the object tool 1201, orientation information and position information of the object 
are acquired. From the pinch tool 1202, two position information of position information of the 
tool and the last position information of the tool are acquired. Thes e This information 
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corr e spond corresponds to the start position and the end position of the pull operation by the 
pinch or the tongs . 

On the basis of twe the acquired position information of the tool, calculation processing 
1203 ef to calculate a movement quantity of the tool is executed. On the basis of the calculated 
movement quantity and position information and orientation information of the object, 
calculation processing 1204 of to calculate a deformation quantity of the object is executed. 

Further, on the basis of the calculation processing result of this the deformation quantity, 
change processing ef to modify the attribute information? {i.e., deformation processing 1205) is 
executed. In combination with these this processing, rendering processing 1206 as proc e ssing 
for generating a display picture with respect to the picture display unit 1 10 is executed. Thus, a 
rendering processing result is stored into the memory for the display T {i.e., the frame memory 
104 of FIG. 2 Figure 2) and is displayed on the picture display unit 110. 

FiGr Figure 13 is a flow for e xplaining flowchart illustrating processing in the cas e wher e 
the using a brush tool as a coloring tool is appli e d to the object to b e e dit e d to apply coloring to 
the displayed object . Step SI 301 is a step for obtaining position/orientation data on the three- 
dimensional space of "brush" us e d as a tool. Such the brush used as the coloring tool. As 
previously describe, such data are obtained on the basis of respective sensor outputs in 
accordance with kiftd the typ e of input unit utilize d, i.e., tabl e ts, magnetic sonsors and ultrasonic 
s e nsors as previously d e scrib e d. . 

Th e n, at step SI 302, In step SI 302, a space relationship/relative position between the 
displayed object to b e e dited and the editing tool is determined from the position/orientation data 
of the object to b e e dit e d which has b ee n acquir e d at th e pr e viously m e ntion e d st e p S801 of FIG. 
S-aftd -displaved object (acquired in step S801 of Figure 8) and the position/orientation data of the 
editing tool "brush" which has boen d e termin e d at th e st e p SI 301. b rush as the editing tool 
(obtained in step SI 301). 

Th e n, at stop SI 303, judgm e nt In step S 1303, a determination is made as to whether or 
not deformation proc e ssing of the displayed object to b e edit e d using by the brush as the editing 
tool "bruGh" is execut e d is brush has been carried out. Such judgm e nt The determination may be 
such that, ergr for example , similarly to the previously described pull tool "pinch", pinch, a 
switch is provided at the brush to detect the push-down of switch, or th e r e the switch. 
Alternatively, a configuration may be employed such a configuration to carry out coloring by the 
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brush when the distance between the displayed object to b e e dit e d and the editing tool which is 
d e t e rmin e d at th e Gt e p S1302 (determined in step S1302) is less than a predetermined distance. 

At the subsequent step SI 304, coloring processing with respect to the object is executed. 
¥he In an embodiment, the operator sets in advanc e the color used in this instance step in 
advance . After coloring processing of the displayed object to b e e dit e d is executed at a the step 
S13CM, rewritten attribute data of the displayed object to b e e dit e d of including the colored result 
is stored iate in the data memory 103 or the external storage unit 106 {shown in FIG. 2 at Figure 
2) in step SI 305. 

For suppl e m e nting In order to supplement the explanation of the flowchart shown in HGr 
Figure 13, explanation will be furth e r giv e n in conn e ction with proc e ssing in th e cas e wh e r e 
coloring tool "brush"is furth e r us e d given w ith reference to the data flow diagram of FIG. 14 
Figure 14 using the brush as a coloring tool . In the data flow of HGr Figure 14, processing is 
started by operation of an object tool (sensor) 1401 as actor object and a brush tool (sensor) 1402 
as an actor object . 

From the object tool 1401, orientation information and position information of the object 
are r e quir e d acquired . From the brush tool 1402, position information of the tool is acquired. 

Further, from the shape information of the object which is the attribute data of the 
displayed object to b e e dit e d which has been acquired in advance? and the orientation 
information and the position information of the object and the position information of the tool 
which are stored in the data memory 103 of FIGr Figure 2, calculation processing 1403 ef to 
calculate the coloring range is executed. On the basis of this calculation result and the surface 
attribute information of the object, chang e processing ef for changing the attribute information? 
(i.e., coloring processing 1404) is executed. In combination with these this processing, rendering 
processing 1405 as proc e ssing for generating a display picture with respect to the picture display 
unit 110 is executed, and a rendering processing result is stored into memory for the display? 
(i.e., the frame memory 104 of FiGrl Figure 2) and is displayed on the picture display unit 110. 

¥iGr Figure 1 5 is a flow for e xplaining flowchart illustrating processing in th e cas e wh e r e 
the using a spray tool as a coloring tool is appli e d to th e obj e ct to b e e dit e d to apply coloring to 
the displayed object . Step SI 501 is step for obtaining position/orientation data in the three- 
dimensional space of "sprav" us e d as the spray (i.e., the spray tool having the shape of a can of 
spray paint) used as a coloring tool. As previously described, such data are data obtained on the 
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basis of respective sensor outputs in accordance with kiftd the type of input unit utilized r 4re7 T 
tabl e ts, magn e tic s e nsors and ultrasonic s e nsors. . 

Th e n, at In step SI 502, the positional relationship of the action range between the 
displayed object to b e e dit e d and the spray tool is determined from position/orientation data of 
the obj e ct to b e e dit e d which has boon acquir e d at th e pr e viously mention e d step S801 of FIG. 8 
displayed object (acquired in step S801 of Figure 8) and position/orientation data of e diting tool 
"spray" determin e d at tho st e p SI 501. the spray tool as the editing tool (obtained in step SI 501). 

Th e n, at st e p SI 503, In step SI 503, a determination is made as to whether or not the 
positional relationship of the action range between the displayed object to b e e dited and the spray 
which has b ee n d e t e rmined at tho st e p SI 502 tool (determined in step SI 502) falls within the 
action area of e diting the spray tool "spray" is judg e d. At as the editing too rspray", . In an 
embodiment, the area which can be colored by that the spray tool is set in advance. In ord e r that 
For the displayed object to be e dit e d is colored, th e r e r e sults th e condition wh e re the positional 
relationship of action range between the obj e ct to b e e dit e d displayed and the spray feHs tool 
must fall within the action area of the spray tool as the editing too f'spray". In this cas e . Thus , 
the action area of the spray is s e t as attribut e of tool in advanc e , tool is set in advance as attribute 
data of the editing tool. 

It is to b e not e d that, in th e spray tool th e action ar e a is s e t as In an embodiment, the 
action area of the spray tool is set as a conical area 1701 in which predetermined points of the 
spray tool are caused to be vertices as indicated by ¥iGr Figure 17 A, for example, and a coloring 
range 1702 with respect to an object is set within the action area on the object 1704 surface as 
shown in WGr Figure 17B. Push button 1703 is provided at on the spray tool, and the coloring 
range 1702 is set on the basis of the positional relationship at the time point when the push 
button 1703 is pushed down. 

When it is judg e d determined at the step SI 503 that the corresponding area is outside the 
action area of the editing spray tool "spray" , coloring processing by the spray tool is not 
executed. As a result, processing is completed. Thus, the routine is returned to the start position. 
When it is judg e d at th e determined in step SI 503 that the corresponding area is inside the action 
area of the e diting spray tool "spray" , the action area is displayed on the picture display unit 1 10 
at step SI 504. 
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Th e n, at In step SI 505, the switch of the e diting spray tool "spray" is pushed down, and a 
determination is made as to whether or not the switch is turned ON is judg e d. With tho fact that 
th e switch hag b ee n turn e d ON being as th e condition. . The switch being ON initiates coloring 
processing with respect to the displayed object is oxecuted at in step SI 506. : Fhe In an 
embodiment, the operator sets in advanc e the color used in this instanc e step in advance . After 
coloring processing of the displayed object to b e e dit e d is executed at in step SI 506, rewritten 
attribute data of the displayed object to b e e dit e d of including the colored result is stored into the 
data memory 103 or the external storage unit 106 {shown in FIG. 2 at Figure 2) in step SI 507. 

For supplem e nting tho d e scription In order to supplement the explanation of the N 
flowchart shown in HGr Figure 15, explanation will be furth e r giv e n in conn e ction with 
proc e ssing in th e cas e wh e r e the coloring tool "spray" is used given with reference to the data 
flow diagram of FIG. 16 Figure 16 using the spray tool as a coloring tool . In the data flow of 
FIGr Figure 16, processing is started by operation of the object tool (sensor) 1601 as actor object 
and the spray tool (sensor) 1602 as an actor object . 

From the object tool 1601, orientation information and position information of the object 
are acquired. From the spray tool 1602, position information and orientation information of the 
tool are acquired. 

On the basis of the position information and the orientation information of the spray tool, 
calculation processing 1603 ef to calculate the action area which can be colored by the spray tool 
is executed. 

Further, calculation processing 1604 ef to calculate the coloring range is carried out on 
the basis of the shape information of the object which is the attribute data of the displayed object 
to be e dit e d which has been acquired in advance? and the orientation information and the position 
information of the object T and the calculated action area of the tool spray which are stored in the 
data memory 103 of FIG. 2, and chang e processing of Figure 2. Processing to modify the 
attribute information T (i.e., coloring processing 1605) is executed on the basis of this calculation. 
In combination with these this processing, rendering processing 1606 as proc e ssing for 
generating a display picture with respect to the picture display unit 110 is executed, and a 
rendering processing result is stored into the memory for the display? (i.e., the frame memory 
104 of HGr4 Figure 2) and is displayed on the picture display unit 110. 
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As stated above, in accordance with the a three-dimensional model processing apparatus 
of this according to an embodiment of the present invention, the operator uses the object tool 
corresponding to the displayed object to b e e dit e d and various dedicated tools to execute a 
change op e ration of in the relative position between both tools, whereby the processing unit 
executes processing set determined in advance in accordanc e with by the dedicated tool selected . 
As a result, processing corresponding to the respective editing tools such as deformation 
coloring, e tc., i. e ., and the like (i.e., an attribute change) is carried out with respect to the obj e ct 
to b e e dit e d displayed on th e pictur e display unit, and object, and the display of its result is 
executed. Accordingly, the operator can execute various processing with such a feeling to 
directly implement processing to the displayed object to bo e dit e d on pictur e. 

Explanation will new be given in connection with th e mor e a practical e xampl e 
embodiment of an application in th e of a three-dimensional model processing apparatus of this 
according to the present invention. Th e e xampl e wh e r e th e In an embodiment, a three- 
dimensional model processing apparatus of this according to the present invention is as applied 
to on-line shopping using a communication network such as the internetreter or the like is shown 
in FIGS. 18Ato 18C. 

The In an embodiment, a three-dimensional model processing apparatus of this invention 
is constitut e d contained within, ergr for example , personal computers of users to display, on 
display 1801, goods from a provider who provides on-line shopping. Such goods display may be 
carried out by storing three-dimensional picture data relating to the goods provided from the 
provider through a network into a storage unit such as a memory, a hard disk, eter and the like 
within the computer of the user to read out such datar-e f. Alternatively, goods display may be 
carried out by using data stored on a medium, e .g., CD or for example, CD, DVD, etGr or the like 
that the goods provider provides. 

The user who studies the goods displays displayed as a three-dimensional picture ef 
goods on the display 1801 to hold holds , by one hand, the object tool (goods m e taphor) 1802 
corresponding to the display goods and to hold holds , by the other hand, the operation tool (teet 
m e taphor) 1803 for executing various processing with respect to the goods. Us e r who studi e s A 
user studying the goods can relatively operate goods m e taphor the object tool 1802 and the 
operation tool m e taphor 1803 to execute processing corresponding to respective tool m e taphors 
operation tools with respect to the goods displayed on the display. For example, by changing the 
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angle/orientation of goods m e taphor the object tool 1802, the article displayed on display is 
displayed in the state where the angle/orientation is changed in a manner from HGr Figure 18A 
to f?Gr Figure 18B. Thus, the user can observe the goods or articles from all angles. 

Further, for example, the a previously described push tool ie may be used as the operation 
tool m e taphor 1803, and a push operation is may be executed by the tool which pushes each 
operation button portions of goods m e taphor of the object tool corresponding to , for example, a 
video camera which is articl e bv th e push tool . Thus, as shown in ¥iGr Figure 18C, th e r e by 
making it it is possible to employ such a configuration that an operation button portion of the 
video camera on the display 1801 is pushed down se such that processing corresponding to push 
down op e rations of e ach the depressed operation button is executed on the display. 

This is r e aliz e d In an embodiment, each operation button may be depressed and executed 
on the display by employing a configuration such that each switch position of the displayed 
object to b e e dit e d is specified on the basis of position information of the push tool to execute a 
processing program corresponding to the specified switch. By executing the program e x e cution 
corresponding to the depressed switch, operation display of the goods within displayed in display 
1801 is may be carried out. In a manner as stated above, a user can actually obtain a feeling te of 
practically hand or hold handling or holding an article to op e rat e and operating it by relative 
operation between goods m e taphor and tool m e taphor the object tool and the operation tool . 

Further, explanation will be given in connection with a different mor e practical e xampl e 
embodiment of an application in th e of a three-dimensional model processing apparatus of this 
according to the present invention. ¥iGr Figure 19 constitutes the a three-dimensional model 
processing apparatus of this according to an embodiment of the present invention within, erg^ for 
example, a personal computer to display a virtual pet on a display 1901 of the personal computer 
te-hel d. An operator holds , by one hand, an object tool 1902 corresponding to the display of the 
virtual pet and to hold holds , by the other hand, various editing tools 1903 to op e rat e . The 
operator operates the editing tool 1903 with respect to the object tool 1902 corr e sponding to p e t 
to th e r e by to execute various processing. 

For example, th e r e may b e e mploy e d a configuration such that a a microphone tool may 
be configured as the editing tool. When the microphone tool is caused to be close to obj e ct tool 
1902 corresponding to p e t with th e e diting tool 1903 b e ing as microphon e tool so that the object 
tool 1902, the displayed pet outputs voice through a speaker. This is realized by executing a 
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subprogram set in correspondence with the microphone tool in tho cas e wh e r e such that when the 
distance between the microphone tool and the object tool 1902 ar e caused to be cloo e go that 
distanc e ther e b e tw ee n is below a predetermined value to e x e cut e ^ processing is executed to 
output voice. Further, a setting may b e mad e go as to e x e cut e included that executes processing 
to change the tone of voice on the basis of the orientation information of the object tool at this 

■f it-vm 

In addition In an embodiment , various tools such as kennel, feed, ete? and the like are set 
as the editing tool to set run subprograms to carry out execution in corr e spond e nc e with 
resp e ctiv e tools so that various processing can b e e x e cut e d in accordance with kifid the various 
types of tools. 

As, ergr for example , processing programs shown in FIGS. 8, 9, 11, 13, 15, e tc. Figures 8- 
16 in the three-dimensional model processing apparatus of this invention, a program stored in the 
program memory 102 shown in HGr Figure 2 may be applied, or there a configuration may be 
employed configuration in which a program is stored in various memory media such as CD- 
ROM, CD-R, CD-RW, DVD-RAM, DVD-RW, DVD+RW, MO, hard disk or floppy disk, otc. 
For e xampl e , in th e cas e of th e pr e viously goods o e lling by goods provid e r, e tc., th e r e may b e 
e mploy e d a configuration such that proc e ssing program is , floppy disk, the like and 
combinations thereof. For example, in an embodiment of goods being sold by a goods provider, 
a processing program may be stored in combination on CD wh e r e thr ee dim e nsional data of 
goods as goods catalog is stor e d to off e r it to us e r. In this cas e , in th e cas e wh e r e th e us e r s e ts a 
CD with a goods catalog to offer it to a user. Thus, the user sets the CD to display goods on a 
display to execute processing using various editing tools, and a processing program 
corresponding to the selected tool is called and is executed. As a result, processing is carried out 
in accordance with the selected tool. Thus, configuration an embodiment for allowing goods 
displayed on a display to execute various operations is realized. 

As stated above, in accordance with th e thr ee dim e nsional mod e l proc e ssing apparatus 
and th e thr ee dim e nsional mod e l proc e ssing m e thod of this inv e ntion, th e r e is e mploy e d th e 
configuration in which op e rator op e ratos various editing tools to e x e cut e processing s e t in 
accordanco with corr e sponding tool. Accordingly, th e op e rator can e x e cut e various proc e ssing 
with a f ee ling as if h e hands or holds tho object to bo edit e d displayed on display to dir e ctly 
op e rat e it. 
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Whil e e xplanation has b ee n giv e n in conn e ction with this inv e ntion with r e f e r e nc e to 
sp e cified e mbodim e nts, it is obviouo that th e p e rson who is drill e d in the art can attain 
modification and/or substitution of th e e mbodim e nts within th e scope which do e s not d e part 
from th e gist. Nam e ly, this inv e ntion has b ee n disclos e d in th e form of e x e mplification, and 
should not b e r e strictiv e ly int e rpr e t e d. In addition, e mbodim e nts constitut e d by combining th e 
abov e d e scrib e d e mbodim e nts as occasion d e mands ar e also included within th e scop e of this 
inv e ntion. For judging th e gist of this inv e ntion, th e column of th e app e nd e d pat e nt claims 
should b e tak e n into consideration. 

Industrial Applicability 

As d e scrib e d abov e , in accordanc e with th e thr ee dim e nsional modol proc e ssing 
apparatus and th e thr ee dim e nsional mod e l proc e ssing m e thod according to this inv e ntion, th e r e 
is e mploy e d th e configuration that an op e rator op e rat e s various e diting tools to e x e cut e 
proc e ssing s e t in accordanc e with th e corr e sponding tool. Accordingly, th e operator can e x e cut e 
various processing with a f ee ling as if h e or sh e dir e ctly op e rat e s th e obj e ct to b e e dited 
displayed on th e display. 

an embodiment of a three-dimensional model processing apparatus and a three- 
dimensional model processing method of the present invention, there a configuration may be 
employed in which an operator operates various editing tools to execute processing determined 
by a corresponding tool. Accordingly, the operator can execute various processing with a feeling 
as if he actually handles or holds the displayed object. 

It should be understood that various changes and modifications to the presently preferred 
embodiments described herein will be apparent to those skilled in the art. Such changes and 
modifications can be made without departing from the spirit and scope of the present invention 
and without diminishing its intended advantages. It is therefore intended that such changes and 
modifications be covered by the appended claims. 
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ABSTRACT 

A thr ee dimensional mod e l proc e ssing apparatus which can more r e ally carry out chang e 
proc e ssing of shap e , color, e ta with r e sp e ct to th e thr ee dim e nsional object display e d on display, 
and an op e rator operat e s th e obj e ct tool in which position chang e and ori e ntation chang e can b e 
mad e and an e diting tool in which r e lativ e distanc e with r e sp e ct to tho obj e ct tool can b e chang e d 
to r e lativ e ly mov e both tools so that th e r e lativ e position b e tw ee n th e object tool and th e e diting 
tool is d e t e ct e d to ex e cut e proc e ssing, e .g., shap e chang e or coloring proc e ssing, e tc. s e t in 
corr e spond e nc e with th e e diting tool on th e basis of th e d e t e ct e d r e lativ e position information 
betw ee n both tools to chang e shap e , color, e tc. by attribut e chang e of th e obj e ct to b e e dit e d 
display e d on display to display it. 

ABSTRACT OF THE DISCLOSURE 

A three-dimensional model processing apparatus is provided. The three-dimensional 
model processing apparatus can more really carry out a processing change of shape, colon and 
the like with respect to a three-dimensional object being displayed on a display by allowing an 
operator to operate an object tool in which a position change and an orientation change can be 
made and an editing tool in which relative distance with respect to the object tool can be changed 
to relatively move both tools such that the relative position between the object tool and the 
editing tool is detected to execute processing corresponding to, for example, shape change or 
color processing determined by the corresponding editing tool thereby modifying attribute data 
of the object being displayed. 
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